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ABSTRACT 

Since the measurement of school achievement involves 
the administration of achievement tests to various grades on various 
subjects, both grade level and subject matter contribute to 
wi thin-school achievement variations. To determine whether 
achievement test scores vary most among different fields within a 
grade level, or within fields among different grades, achievement 
test scores from 88 Israeli elementary schools were examined. Test 
scores of the resulting 15 combinations of grades by subjects were 
averaged. The average wi thin-school variance was 25% of the total 
across-schools variance, showing notable heterogeneity. Computation 
of the within-school range for the 15 means showed 55% to exceed one 
standard deviation. Further analyses showed more similarity in mean 
achievements of the same grade level in different fields than of 
different grade levels in the same field. The first analysis, 
involving a within-school anlaysis, had three components: between 
grades, between subject areas, and the interaction. The second, 
nonmetric analysis of the correlation matrix between the 15 school 
means was conducted using smallest space analysis and ADDTREE. 
Furthermore, in 88% of the schools where over 51% of the 
within-school variance was caused by a single component, that 
component was grade level. (M6D) 
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The last 20 years witnessed an ever increasing awareness of the "unit 
of analysis" problem in general, and of the school, as the appropriate 
object of inquiry in educational research, in particular. The smallest 
autonomous unit in most educational systems, the school is also the smallest 
unit for many policy-making decisions. 

Among the various school characteristics, school achievement is the 
most prominent. Since instruction is usually organized according to grade 
level and subject matter, the measurement of school achievement involves the 
administration of a different achievement test for each combination of grade 
and subject matter. Corresponding to each test there is a school mean score. 
Therefore, a proper conceptualization of school achievement involves the set 
or profile of the grade by subject matter school means. Obviously, the raw 
.nean test scores of the same school on different tests are not directly 
comparable. However, ipsative comparisons are possible between the z-scores 
corresponding to the various school means in the relevant population of 
schools. 

A relevant question concerning school achievement profiles refers to 
their heterogeneity or scatter. To what extent are schools homogeneous in 
terms of achievement? In addition to its intrinsic significance, this 
qu-otion also has practical implications for policy decisions. For example, 
noticeable heterogeneity of school achievement profiles may raise doubt 
concerning the status of schools as the smallest units for the differential 
allocation of resources. 

A related question concerns the relative contributions of grade level 
and subject matter to the within school variation in achievement. Put 
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differently, the question is which is usually higher: the within grade level 
variation of the school means in different fields or the within field 
variation of the school means for different grade levels? 

The study reported here was concerned with the empirical investigation 
of these two questions. The sample consisted of grades 1,2,4 and 6 in 58 
elementary schools in a large sc*le study of the Israeli elementary 
education (Minkowich, Davis & Bashi, 1982), in which students were 
administered several achievement tests* The grade level by subject matter 
test matrix is presented in Table 1. 



Table 1 about here 



88 schools had complete information. Test scores for each of the 15 
combinations of grade by subject matter were averaged for each of these 
schools. Each of the 15 means was standardized in the entire sample of 
schools with mean zero and unit standard deviation. 

RESULTS 

One definition of the heterogeneity of school achievement profiles is 
in terms of the within school variance of the 15 standardized means. This 
was computed for each of the 88 schools. The average within school variance 
was found to be 25% of the total, across schools variance, with a standard 
deviation of 18%. This is interpreted as evidence for the noticeable 
heterogeneity of the achievement profiles of a considerable proportion of 
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schools. Another way to illustrate this heterogeneity is by means of the 
within school range of the 15 standardized means (see Table 2). 

Table 2 about here 



In 55% of the schools, this range exceeded one standard deviation. 

I turn now to the second question. Given that school achievement varies 
considerably between the different combinations of grade level and subject 
matter, which are more similar: the relative achievements of the same school 
for different grade levels in the same subject matter or the relative 
achievements of the same grade level in different subject matters? Two 
analyses, which are based on the same data, were performed: 

1) A within schools analysis, involving the decomposition of the within 
school variance of the 15 grade by subject matter means into three 
components: between grades, between subject matters, and the interaction 
between grade and subject matter. 

2) An overall nonmetric analysis of the correlation matrix between the 
15 schools means in the sample of 88 schools. This was done by two methods: 
Smallest Space Analysis (SSA, Guttman, 1968) and ADDTREE (Sattath & Tver sky, 
1977). 

Both analyses revealed the same phenomenon: the mean achievements of 
the same grade level in different fields were considerably more similar than 
the mean achievements of different grade levels in the same field. In within 
school terms this is reflected in the fact that on tho average across 
schools, only 23% of the within school variance lies between subject 
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matters, vs. 44% between grade levels (see Table 3). A related, perhaps even 
more impressive finding is that in 88% of the 44 schools in which at least 
51% of the within school variance was attributable to a single component, 
this component was grade level, rather than subject matter. 



Table 3 about here 



From an overall, across schools point of view the same phenomenon is 
reflected in correlations which are higher, on the average, between subject 
matters, within the same grade level, than within subject matter, between 
grade levels. The correlation matrix among the 15 school means is given in 
Table 4. 



Table 4 about here 



Guttman and Lingoes 1 SSA-I (see Figure 1) and Sattath and Tversky's 
ADDTREE (see Figure 2) provide two different representations of the 
correlation matrix. Both illustrate the greater similarity between the 
relative achievements of the same grade level in different fields than 
between the relative achievements of different grade levels in the same 
field. 



Figure 1 and 2 about here 
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A practical implication of these findings concerns the sampling of 
grade levels and subject matter for the purpose of estimating the elevation 
of the school achievement profiles, usually defined in terms of the average 
of the grade by subject matter means. They point to sampling of grade levels 
rather than subject matters as the preferred approach to the estimation of 
the overall index of school achievement, However, these findings may be 
highly specific to the educational system, grade levels, and subject matters 
investigated. Extensive research is needed to determine generalizability 
over these dimensions. 
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Table 1: The grade level x subject matter test matrix. 



| Grade 
| Level 


! Reading* 
! Comprehension 
! I 


Reading* | 
Comprehension! 
II j 


Math 


! Geography | 


Science j 


| 1 


i LI 


Rl ! 


Ml 






! 2 




R2 | 


M2 






! A 


! LA 


RA j 


MA 


! JA j 


TA ! 


1 6 


i L6 


R6 ! 


M6 


! J6 ! 


T6 i 



* L and R are two different Reading Comprehension tests. 
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Table ?.: The within-school range of the 15 standardized school means. 



The within-school range* 


Percentage of schools 


0 - .5 


23 


.5-1.0 


21 


1.0-1.5 


30 


1.5-2.0 


12 


2.0-2.5 


5 


2.5-6.0 


9 


Total 


100 



* Standard deviation units. 
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Table 3: The distribution (percentages of schools) of the between -grade 
levels, between subject matters and residual components of the 
within school variance. 



Proportion of 


Between 


Between 


Residual 


within school 


grades 


subject matters 


(Interaction) 


Vclt XcUlCc 








0-.25 


27 


68 


35 


.26-. 50 


39 


26 


49 


.51-. 75 


25 


6 


16 


.76-1.0 


9 






Total 


100 


100 


100 


Mean 


.44 


.23 


.33 


S.D. 


.23 


.15 


.18 
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Table 4: The correlations between the 15 grade by subject matter school means. 











Grade 


1 


! Grade 2 






Grade 


A 






Grade 6 




! Grade 


! Test 


Symbol! 


LI 


Ml 


Rl 


! M2 


R2 


! M 


LA 


MA 


RA 


TA 


! J6 


L6 M6 


R6 


; 1 


{Math 


Ml | 


.71 


- 


























'Reading Comprehension 


Rl j 


.88 


.76 


- 






















• 2 


jMath 


M2 ! 


.57 


.73 


.6A 






















}■■ 


{Reading Comprehension 


H2 ! 


.62 


.72 


.70 


.85 






















'Geography | 


J4 ! 


.55 


.65 


.55 


.65 


.78 


- 


















| Reading Comprehension! 


L4 ; 


.65 


.7A 


.69 


.76 


.8A 


.82 
















! 4 


Math j 


MA j 


.57 


.65 


.65 


.73 


.83 ! 


.82 


.89 
















Reading Comprehension! 


R4 j 


.61 


.7A 


.66 ; 


.73 


.82 i 


.81 


.89 


.87 














Nature j 


TA j 


.48 


.65 


.55 1 


.73 


.80 | 


.78 


.85 


.85 


.78 












Geography [ 


J6 [ 


.51 


.62 


.5A ! 


.68 


.76 ! 


.7A 


.82 


.77 


.75 


.79 ! 










Reading Comprehension! 


L6 j 


.60 


.68 


.62 ! 


.75 


.80 ! 


.75 


.89 


.79 


.80 


.78 ! 


.91 






! 6 ; 


Math ! 


M6 j 


.57 


.62 


.58 ! 


.68 


.75 ! 


.7A 


.82 


.80 


.77 


.7A ] 


.89 


.90 






Reading Comprehension! 


R6 j 


.57 


.68 


.60 ! 


.72 


.79 ! 


.72 


.8A 


.74 


.76 


.77 j 


.89 


.91 .87 






Nature { 


T6 j 


.A7 


.60 


.52 j 


.69 


.75 j 


.72 


.80 


.79 


.7A 


.79 j 


.85 


.84 .84 


.77 i 
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Figure 1: The smallest space analysis (SSA-I) of the similarity relations 
among the 15 grade by subject matter school means. 
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M4 



J4 



R4 



R2 



Coefficient of Allienation* .C 
Kruskal Stress « .( 
Linearly Accounted Variance 5 .£ 



M2 



M1 
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gure 2: The Additive Tree (ADDTREE) representation of the similarity relations 
among the 15 grade by subject matter school means. 
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